Reliable Off-policy Evaluation for Reinforcement Learning

In a sequential decision-making problem, off-policy evaluation estimates the expected cumulative reward of a target policy using logged trajectory data generated from a different behavior policy, without execution of the target policy. Reinforcement learning in high-stake environments, such as healthcare and education, is often limited to off-policy settings due to safety or ethical concerns, or inability of exploration. Hence it is imperative to quantify the uncertainty of the off-policy estimate before deployment of the target policy. In this paper, we propose a novel framework that provides robust and optimistic cumulative reward estimates using one or multiple logged trajectories data. Leveraging methodologies from distributionally robust optimization, we show that with proper selection of the size of the distributional uncertainty set, these estimates serve as confidence bounds with non-asymptotic and asymptotic guarantees under stochastic or adversarial environments. Our results are also generalized to batch reinforcement learning and are supported by empirical analysis.Comment: 39 pages, 4 figure

Histone Deacetylase 1 (HDAC1) Negatively Regulates Thermogenic Program in Brown Adipocytes via Coordinated Regulation of H3K27 Deacetylation and Methylation

Inhibiting class I histone deacetylases (HDACs) increases energy expenditure, reduces adiposity and improves insulin sensitivity in obese mice. However, the precise mechanism is poorly understood. Here, we demonstrate that HDAC1 is a negative regulator of brown adipocyte thermogenic program. HDAC1 level is lower in mouse brown fat (BAT) than white fat, is suppressed in mouse BAT during cold exposure or β3-adrenergic stimulation, and is down-regulated during brown adipocyte differentiation. Remarkably, overexpressing HDAC1 profoundly blocks, whereas deleting HDAC1 significantly enhances β-adrenergic activation-induced BAT-specific gene expression in brown adipocytes. β-adrenergic activation in brown adipocytes results in a dissociation of HDAC1 from promoters of BAT-specific genes, including uncoupling protein 1 (UCP1) and peroxisome proliferator-activated receptor γ co-activator 1α (PGC1α), leading to increased acetylation of histone H3 lysine 27 (H3K27), an epigenetic mark of gene activation. This is followed by dissociation of the polycomb repressive complexes, including the H3K27 methyltransferase enhancer of zeste homologue (EZH2), suppressor of zeste 12 (SUZ12), and ring finger protein 2 (RNF2) from, and concomitant recruitment of H3K27 demethylase ubiquitously transcribed tetratricopeptide repeat on chromosome X (UTX) to UCP1 and PGC1α promoters, leading to decreased H3K27 trimethylation, a histone transcriptional repression mark. Thus, HDAC1 negatively regulates brown adipocyte thermogenic program, and inhibiting HDAC1 promotes BAT-specific gene expression through a coordinated control of increased acetylation and decreased methylation of H3K27, thereby switching transcriptional repressive state to active state at the promoters of UCP1 and PGC1α. Targeting HDAC1 may be beneficial in prevention and treatment of obesity by enhancing BAT thermogenesis

Histone Deacetylase 1 (HDAC1) Negatively Regulates Thermogenic Program in Brown Adipocytes via Coordinated Regulation of H3K27 Deacetylation and Methylation

Inhibiting class I histone deacetylases (HDACs) increases energy expenditure, reduces adiposity and improves insulin sensitivity in obese mice. However, the precise mechanism is poorly understood. Here, we demonstrate that HDAC1 is a negative regulator of brown adipocyte thermogenic program. HDAC1 level is lower in mouse brown fat (BAT) than white fat, is suppressed in mouse BAT during cold exposure or β3-adrenergic stimulation, and is down-regulated during brown adipocyte differentiation. Remarkably, overexpressing HDAC1 profoundly blocks, whereas deleting HDAC1 significantly enhances β-adrenergic activation-induced BAT-specific gene expression in brown adipocytes. β-adrenergic activation in brown adipocytes results in a dissociation of HDAC1 from promoters of BAT-specific genes, including uncoupling protein 1 (UCP1) and peroxisome proliferator-activated receptor γ co-activator 1α (PGC1α), leading to increased acetylation of histone H3 lysine 27 (H3K27), an epigenetic mark of gene activation. This is followed by dissociation of the polycomb repressive complexes, including the H3K27 methyltransferase enhancer of zeste homologue (EZH2), suppressor of zeste 12 (SUZ12), and ring finger protein 2 (RNF2) from, and concomitant recruitment of H3K27 demethylase ubiquitously transcribed tetratricopeptide repeat on chromosome X (UTX) to UCP1 and PGC1α promoters, leading to decreased H3K27 trimethylation, a histone transcriptional repression mark. Thus, HDAC1 negatively regulates brown adipocyte thermogenic program, and inhibiting HDAC1 promotes BAT-specific gene expression through a coordinated control of increased acetylation and decreased methylation of H3K27, thereby switching transcriptional repressive state to active state at the promoters of UCP1 and PGC1α. Targeting HDAC1 may be beneficial in prevention and treatment of obesity by enhancing BAT thermogenesis

A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics

In today's competitive and fast-evolving business environment, it is a critical time for organizations to rethink how to make talent-related decisions in a quantitative manner. Indeed, the recent development of Big Data and Artificial Intelligence (AI) techniques have revolutionized human resource management. The availability of large-scale talent and management-related data provides unparalleled opportunities for business leaders to comprehend organizational behaviors and gain tangible knowledge from a data science perspective, which in turn delivers intelligence for real-time decision-making and effective talent management at work for their organizations. In the last decade, talent analytics has emerged as a promising field in applied data science for human resource management, garnering significant attention from AI communities and inspiring numerous research efforts. To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management. Specifically, we first provide the background knowledge of talent analytics and categorize various pertinent data. Subsequently, we offer a comprehensive taxonomy of relevant research efforts, categorized based on three distinct application-driven scenarios: talent management, organization management, and labor market analysis. In conclusion, we summarize the open challenges and potential prospects for future research directions in the domain of AI-driven talent analytics.Comment: 30 pages, 15 figure

Dependences and volatility spillovers between the oil and stock markets: New evidence from the copula and VAR-BEKK-GARCH models

This paper examines the dynamic relationship between the oil market and stock markets from two perspectives: dependence between the crude oil market (WTI) and stock markets of the US and China, and volatility spillovers between them during 1991-2016. We further analyze structural breaks of market dependences and consider the extent of their influence on such relationships. Our vine-copula results show that the dependences between the three paired markets, WTI-US, WTI-China and US-China, vary dynamically across the six identified structural break periods. In particular, the dependence between WTI-US is stronger and more volatile than that between WTI-China during most of the periods. The dependence between US-China remains at a lower level in the earlier periods, but increases in the final period. Our VAR-BEKK-GARCH results demonstrate distinctive volatility spillovers across these periods, with varying directionality, in response to the structural changes. Overall, our results indicate that the oil market stimulates rapid and continual fluctuations in market dependences, which become manifest most acutely in the aftermath of the Financial Crisis of 2007-08, demonstrating the increasing interdependence between the oil and stock markets. Further, the growing influence of China on the dynamics of these relationships, in the period following the Great Recession, presents evidence that it begins to assume an increasingly important role in global economic recovery

Influence of Disturbance on Soil Respiration in Biologically Crusted Soil during the Dry Season

Soil respiration (Rs) is a major pathway for carbon cycling and is a complex process involving abiotic and biotic factors. Biological soil crusts (BSCs) are a key biotic component of desert ecosystems worldwide. In desert ecosystems, soils are protected from surface disturbance by BSCs, but it is unknown whether Rs is affected by disturbance of this crust layer. We measured Rs in three types of disturbed and undisturbed crusted soils (algae, lichen, and moss), as well as bare land from April to August, 2010, in Mu Us desert, northwest China. Rs was similar among undisturbed soils but increased significantly in disturbed moss and algae crusted soils. The variation of Rs in undisturbed and disturbed soil was related to soil bulk density. Disturbance also led to changes in soil organic carbon and fine particles contents, including declines of 60–70% in surface soil C and N, relative to predisturbance values. Once BSCs were disturbed, Q10 increased. Our findings indicate that a loss of BSCs cover will lead to greater soil C loss through respiration. Given these results, understanding the disturbance sensitivity impact on Rs could be helpful to modify soil management practices which promote carbon sequestration

Observation of nonrelativistic plaid-like spin splitting in a noncoplanar antiferromagnet

Spatial, momentum and energy separation of electronic spins in condensed matter systems guides the development of novel devices where spin-polarized current is generated and manipulated. Recent attention on a set of previously overlooked symmetry operations in magnetic materials leads to the emergence of a new type of spin splitting besides the well-studied Zeeman, Rashba and Dresselhaus effects, enabling giant and momentum dependent spin polarization of energy bands on selected antiferromagnets independent of relativistic spin-orbit interaction. Despite the ever-growing theoretical predictions, the direct spectroscopic proof of such spin splitting is still lacking. Here, we provide solid spectroscopic and computational evidence for the existence of such materials. In the noncoplanar antiferromagnet MnTe$_2$, the in-plane components of spin are found to be antisymmetric about the high-symmetry planes of the Brillouin zone, comprising a plaid-like spin texture in the antiferromagnetic ground state. Such an unconventional spin pattern, further found to diminish at the high-temperature paramagnetic state, stems from the intrinsic antiferromagnetic order instead of the relativistic spin-orbit coupling. Our finding demonstrates a new type of spin-momentum locking with a nonrelativistic origin, placing antiferromagnetic spintronics on a firm basis and paving the way for studying exotic quantum phenomena in related materials.Comment: Version 2, 30 pages, 4 main figures and 8 supporting figure